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Abstract 

The  question  of  where  to  best  invest  limited 
research  and  development  funds  is  a  major 
concern.  In  missile  defense  the  problem  is 
exacerbated  by  the  diversity  of  technologies  that 
may  be  employed.  Frequently,  subjective 
methods  such  as  the  Delphi  method  and  various 
weighting  schemes  are  employed  to  rank 
choices.  This  paper  introduces  an  objective 
methodology  that  avoids  the  problems 
encountered  in  subjective  weighting  schemes.  It 
utilizes  a  System  Performance  Index  (SPI)  based 
on  the  physics  of  the  problem  as  a  unifying 
measure  for  assessing  and  ranking  diverse 
technologies.  The  SPI  is  coupled  with  five  other 
measures  —  resources,  cost,  return  on 
investment,  technology  readiness  level,  and  time 
of  insertion  —  to  completely  characterize  the 
technology  with  respect  to  a  baseline.  The  SPI 
is  calculated  as  the  summation  of  the  product  of 
the  Probability  of  Zero  Leakage  and  the  Value  of 
the  target  area  over  the  entire  defended  region. 
The  methodology  utilizes  an  abstract  system 
model  that  embodies  the  physics  of  the  problem 
and  operates  with  Critical  Performance 
Parameters  that  can  be  derived  for  the 
technologies  using  well-established  analytical 
models  and  processes.  Available  tools  for 
implementing  the  methodology  and  shortcomings 
are  discussed,  and  the  methodology  is 
demonstrated  through  and  example. 

Introduction 

When  you  can  measure  what  you  are  speaking 
about  and  express  it  in  numbers, 
you  know  something  about  it. 

Lord  Kelvin 

The  question  of  where  to  “best”  invest  limited 
research  and  development  dollars  is  always  a 
major  concern  to  system  managers,  engineers, 
and  scientists.  Ballistic  missile  defense  (BMD) 
technologies  are  broad  and  diverse;  thus,  making 
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comparison  of  choices  complex  and  difficult. 
Frequently,  subjective  methods  such  as  the 
Delphi  method  and  various  weighting  schemes 
are  employed  to  rank  choices.  The  Delphi 
method  attempts  to  draw  upon  the  expertise  of 
experts  in  the  field  to  make  “smart”  choices. 
While  the  Delphi  method  is  useful  for  quickly 
screening  a  large  number  of  choices,  it  is  often 
difficult  to  find  individuals  with  expertise  across 
the  many  technologies  that  may  be  employed 
throughout  a  BMD  system.  Consequently, 
choices  of  investment  strategies  sometimes  look 
more  like  a  beauty  contest  than  objective 
decision-making.  Furthermore,  the  process  may 
become  highly  politicized.  Weighting  schemes 
suffer  from  similar  problems.  In  these  cases 
there  is  the  problem  of  picking  the  weights. 
Usually,  this  is  a  subjective  process  with  little  to 
guide  the  selection  except  the  experience  and 
“feel”  of  the  selector.  This  gives  rise  to  the 
adage,  “He  who  chooses  the  weights  determines 
the  outcome.”  Unless  there  is  a  quantitative 
method  for  picking  the  weights,  the  process  may 
deteriorate  into  a  political  process.  Sometimes, 
complex  simulations  are  constructed  to  compute 
effectiveness.  This  approach,  while  yielding 
good  results,  may  be  costly  and  time  consuming. 
Furthermore,  some  objective  method  of  making 
comparisons  is  still  required. 

Fundamentally,  there  are  three  questions  the 
program  manager  or  system  engineer  wants 
answered  with  respect  to  new  technologies. 
These  are: 

o  How  will  it  help  solve  my  problem? 

o  What  will  it  cost? 

o  When  will  it  be  available? 

The  first  question  can  be  viewed  as  a  system 
performance  issue,  although  some  might  argue 
that  it  includes  affordability  and  availability  as 
well.  We  argue  that  these  are  resource  issues 
and  are  best  considered  as  part  of  the  cost 
question,  which  includes  research  and 
development,  acquisition,  and  operation  and 
support  costs  to  achieve  a  specified  level  of 
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system  performance.  Several  cost  models  1,2  are 
available  to  answer  the  second  question  once  the 
system  element  resources  have  been  quantified. 
Systems  such  as  NMD,  THAAD,  PATRIOT,  and 
MEADS  also  have  lifecycle  cost  (LCC)  models 
that  may  be  used  in  specific  cases.  The  third 
question  is  a  schedule  question  driven  primarily 
by  the  technology  readiness  and  funding 
constraints. 

In  this  paper  an  objective  methodology  is  outlined 
that  supports  decisions,  which  are  driven  by  the 
physics  of  the  problem  rather  than  the  politics  of 
the  problem.  During  the  mid-to-late  1980s,  the 
USASMDC  investigated  and  developed  methods 
and  performance  measures  for  designing, 
evaluating,  and  controlling  large-scale  Strategic 
Defense  Initiative  (SDI)  systems.  This 
methodology  work  ,4,5  demonstrated  the 
effectiveness  of  system  performance  indices  as  a 
methodology  for  optimizing  performance  of 
complex  systems  (and  systems  of  systems)  over 
a  range  of  complex  decisions  and  provides  the 
foundation  for  the  methodology  presented  in  this 
paper.  Results  were  validated  with  data  from 
high  fidelity  testbeds  for  Low  Altitude  Defense 
Systems  and  SDI  Battle  Management/Command, 
Control,  and  Communications  systems.  In  this 
paper  we  build  on  this  foundation  and  extend  the 
methodology  to  assess  and  rank  technology 
choices. 

The  System  Performance  Index 

Previous  work  3,4,5  has  demonstrated  the  use  of  a 
System  Performance  Index  in  optimizing  BMD 
systems.  This  work  chose  the  formulation, 

SPI  =  I  Psi*  V|  (1) 

where,  Psi  is  the  probability  of  survival  of  the  ith 
defended  asset  and  V,  is  the  value  assigned  to 
the  ith  defended  asset.  Summing  over  all  i 
assets  yields  the  expected  surviving  value.  Thus, 
to  find  the  “best”  choice  only  requires  that  we 
make  choices  that  maximize  the  System 
Performance  Index,  SPI.  For  complex  systems, 
computation  of  the  SPI  requires  an  abstract 
model  or  simulation  that  computes  the  probability 
of  survival  based  on  fundamental  physical 
principals.  This  process  requires  algorithms  that 


capture  the  essential  physics  constraining  or 
driving  the  performance  of  the  system  at  a  level 
of  fidelity  that  will  enable  an  accurate  distinction 
to  be  made  between  system  alternatives. 

At  first  glance  this  may  seem  to  be  a  daunting  a 
task,  however,  we  only  have  to  find  the  Psi  for 
each  defended  asset  impacted  by  our  choices.  If 
we  are  considering  new  technologies  for 
incorporation  in  the  BMD  system,  this  is  a 
relatively  small  set.  Thus,  we  are  able  to  compute 
SPI  for  all  choices  and  use  the  results  to  rank 
order  the  choices,  i.e.,  rank  the  technologies. 

When  equation  (1)  was  formulated,  the  threat 
was  counted  in  the  thousands  and  many  of  the 
defended  assets  were  hard  targets.  Today,  the 
threat  is  vastly  different.  The  threat  is  counted  in 
tens  and  most  of  the  targets  are  soft,  i.e.,  cities, 
population  centers,  etc.  Consequently,  today 
there  is  emphasis  on  the  probability  of  zero 
leakage  PZL  as  a  measure  of  performance.  Thus, 
the  SPI  can  be  reformulated  to 

SPI  =  I  PZLi  *  Vi.  (2) 

While  the  equation  (2)  formulation  is  not  precisely 
equivalent  to  equation  (1),  it  is  approximately 
equivalent  assuming  soft  targets  and  relatively 
small,  defended  areas.  With  equation  (2),  we 
now  have  a  formulation  that  can  be  used  to 
optimize  among  available  choices.  In  comparing 
technologies  it  may  be  convenient  to  normalize 
the  SPI  as  follows, 

SPI  =  IPZLi*Vi/SVi  (3) 

to  provide  a  SPI  that  varies  between  0  and  1 . 

Technology  Assessment  and  Ranking 
Methodology 

The  SPI  provides  an  objective  quantitative  means 
of  assessing  and  comparing  the  impact  of  diverse 
technologies  on  system  performance.  We  will 
discuss  an  overall  methodology  for  computing  the 
SPI,  quantifying  cost,  assessing  readiness  for 
insertion  in  the  system,  and  using  the  results  to 
rank  the  technologies. 
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Establish  a  baseline  SPI  using  existing  technologies 
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CPPj  =  Critical  Performance  Parameter(s) 

T#  =  Technology 

SPI  =  System  Performance  Index 

PZL  =  Probability  of  Zero  Leakage 

V;  =  Cell  Value 

R  =  Resource  Matrix 

rk  =  Resource  Quantity 


Figure  1.  Process  for  Establishing  the  Baselines  for  SPI  and  Resources 


Establishing  a  Baseline 

The  methodology  starts  with  the  establishment 
of  a  baseline  system  performance  index, 
SPIbaseiine,  and  associated  resources,  Rbaseiine,  for 
the  baseline  technologies  used  in  the  system. 
This  process  is  illustrated  in  Figure  1.  The 
process  uses  a  System  Model  that  embodies  the 
System  Architecture  and  responds  to  a  Threat 
model  to  compute  the  SPI  and  the  associated  R. 
Threat  evolution  is  an  important  consideration  in 
the  final  technology  ranking  process.  The  SPI 
will  vary  in  response  to  changes  in  threat  over 
time.  A  full  analysis  will  entail  determination  of 
the  SPI  for  different  “threat  epochs” 
corresponding  to  periods  of  time  when  a 
particular  threat  suite  is  projected  to  emerge 
and/or  predominate.  The  key  to  success  with 
this  process  is  a  System  Model  that  represents 
the  system  at  the  right  level  of  abstraction.  It 
must  provide  outputs  that  can  be  used  to 
compute  the  PZl  associated  with  each  defended 
asset  or  area.  The  model  must  embody  the 
basic  physics  of  the  problem  and  operate  with 
Critical  Performance  Parameters  (CPP)  that  can 


be  derived  for  the  technologies  using  well- 
established  analytical  models  and  processes.  A 
typical  analytical  model  might  use  the  radar 
range  equation,  sensor  sensitivity,  etc.  to 
generate  CPPs.  Example  CPPs  include 
detection  range,  discrimination  range, 
interceptor  flyout  fans,  etc.  that  can  be  derived 
from  the  characteristics  of  the  technologies,  T. 

Calculate  SPI  and  R  with  New  Technology 

The  next  step  is  to  calculate  the  SPI  and 
associated  resources,  R,  with  the  new 
technology  inserted.  The  new  technology,  T,  will 
impact  one  or  more  Critical  Performance 
Parameters  (CPP),  which  will  in  turn  be  reflected 
in  the  response  of  the  System  Model  to  the 
Threat.  It  does  not  matter  if  a  new  technology 
generates  an  SPI  value  by  influencing  several 
CPPs  or  by  influencing  a  single  CCP.  In  either 
case  the  influence  of  the  technology  on  the  SPI 
will  be  properly  calculated.  It  is  important  to 
distinguish  between  this  approach  and  weighting 
schemes  that  are  not  driven  by  impact  on 


3 

UNCLASSIFIED 


UNCLASSIFIED 


‘tr 


Insert  New  Technology 


P zl  =  Probability  of  Zero  Leakage 
V;  =  Cell  Value 
R  =  Resource  Matrix 
rk  =  Resource  Quantity 


Figure  2.  Process  for  Calculating  the  SPI  &  R  with  New  Technology 


system  performance.  Weighting  schemes  that 
rely  on  a  preponderance  of  data  or  opinion 
generally  will  not  properly  capture  the 
cumulative  effect  of  a  technology  across  multiple 
CPPs,  and  can  be  a  major  source  of  error  in 
ranking. 

The  overall  process  for  calculating  SPI  and  R  is 
illustrated  in  Figure  2.  The  results  are  a  new 
SPI  and  resource  matrix,  R,  that  can  be 
compared  to  the  established  baseline.  We  now 
have  a  quantitative  measure  of  the  improvement 
offered  by  the  new  technology  and  a  basis  for 
estimating  the  associated  cost. 

Calculate  Cost  and  Return  on  Investment 

The  next  step  in  the  methodology  is  to  estimate 
the  cost  associated  with  the  baseline  and  the 
new  technology.  The  approach  is  illustrated  in 
Figure  3.  The  resource  matrix,  R,  from  the 
System  Model  is  input  to  a  Cost  Model  to  obtain 
an  estimate  of  the  cost  associated  with  the  level 
of  system  performance  denoted  by  the  SPI  for 
each  technological  innovation.  It  also  may  be 
useful  to  normalize  the  cost  to  the  baseline  cost. 
The  cost  model  must  produce  an  estimate  of  the 


total  lifecycle  cost  to  include  Research, 
Development,  Test,  and  Evaluation  (RDT&E); 
System  Acquisition;  and  Operation  and  Support 
(O&S).  This  model  could  be  the  existing 
lifecycle  cost  (LCC)  model  for  the  system,  or  for 
that  matter  several  different  system  LCC  models 
or  element  cost  models  could  be  used  to  support 
the  cost  estimation  requirement.  Note,  the 
previous  Large  Scale  Systems  Technology  work 
discussed  previously  addressed  only  system 
resources  and  did  not  specifically  address 
detailed  cost  modeling.  In  this  respect,  the 
process  outlined  in  this  paragraph  represents  an 
extension  of  that  methodology. 

A  simple  calculation  of  Return  On  Investment 
(ROI),  based  on  the  SPI  and  Cost,  provides 
another  measure  that  is  extremely  useful  in  the 
final  ranking  process  because  it  allows  us  to 
consider  cost  reduction  technologies  and 
performance  enhancing  technologies  in  the 
same  frame  of  reference.  The  ROI  is  computed 
by  dividing  the  SPI  by  the  Cost,  C.  This 
measure  indicates  which  technologies  yield  the 
most  performance  for  an  investment  dollar. 
Although  ROI  cannot  be  used  alone,  it  is  very 
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Input  Resources  RATj+1  into  Cost  Model  &  Calculate  CATj+1 


Cost  Model 


ATj+1 


Compute  Return  on  Investment,  ROIATj+1  =  SPIATj+1  / C ATj+1 


Figure  3.  Process  for  Calculating  Cost  and  Return  on  Investment 


useful  in  ranking  technologies  that  are  above  a 
specified  system  performance  threshold,  and 
can,  therefore,  be  very  useful  in  deciding  where 
to  make  investments. 

Technology  Readiness 

The  question  of  when  the  technology  will  be 
available  is  complex,  comparable  to  predicting 
the  future.  Nine  levels6  7  of  readiness  have  been 
defined  by  NASA  and  adapted  by  the  DOD  in  an 
attempt  to  make  the  process  more  quantifiable 
and  objective.  Although  the  technology 
readiness  levels  (TRLs)  are  objective,  given  a 
standard  interpretation  of  the  definitions,  they  do 
not  answer  the  question  of  when  the  technology 
will  be  available  for  insertion.  However,  the  TRL 
does  provide  a  useful  approach  for  assessing 
where  a  technology  is  in  its  developmental 
lifecycle.  It  is  tempting  to  view  the  TRL  as 
sufficient,  but  it  is  of  limited  value  in  estimating 
the  time  epoch  within  a  system  lifecycle  when 
the  technology  will  be  ready  for  insertion  into  the 
system  because  we  currently  have  no  models  to 
predict  when  the  technology  will  be  ready  for 
insertion  given  its  current  TRL.  Unfortunately, 
today  the  only  way  to  get  this  information  is  for 
the  technologist  to  develop  detailed  schedules 
that  show  when  the  technology  is  ready  for 
insertion.  Any  estimate,  of  course,  is  predicated 


on  the  skill  of  the  technologist  and  the  level  of 
available  funding.  Two  schedules  are  of  interest 
-  constrained  and  unconstrained.  The  first 
assumes  a  level  of  funding  to  mature  the 
technology  that  is  less  than  could  be  fully 
utilized.  The  second  assumes  funding  is  not  an 
issue  and  timelines  are  constrained  only  by  the 
technologist’s  capability  to  mature  the 
technology.  We  argue  that  the  unconstrained 
schedule  is  the  most  important  for  technology 
assessment  and  ranking,  and  that  it  should  be 
used  to  determine  the  Time  of  Insertion  (TOI). 

Models  based  upon  empirical  or  historical  data 
have  been  successfully  developed  and  widely 
used  to  estimate  schedules  for  complex 
systems.  Examples  include  software  estimation 
models  such  as  COCOMO8  and  REVIC9. 
Comparable  models  are  needed  to  estimate 
when  the  technology  will  be  ready  for  insertion 
given  where  it  is  in  the  development  cycle  (i.e., 
its  TRL),  the  type  of  technology,  and  other 
technology  unique  factors.  Techniques  such  as 
those  employed  in  COCOMO  and  REVIC  may 
offer  fruitful  avenues  for  research  in  this  area. 
Until  such  models  are  available,  we  will  be 
forced  to  rely  on  schedules  generated  by  the 
technologists  to  establish  the  TOI. 
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Ranking  the  Technologies 

The  ranking  process  begins  by  ranking  the 
technologies  in  descending  order  from  the 
highest  SPI  to  the  lowest.  The  minimum 
acceptable  system  performance  against  a  threat 
is  used  to  establish  a  threshold  level  of 
performance  and  the  maximum  cost  constraint  is 
used  to  put  a  ceiling  on  the  cost  we  are  willing  to 


support  insertion  to  counter  projected  threats. 
Graphically  this  is  illustrated  in  Figure  4.  For 
capability-based  deployments  TOI  simply 
represents  the  earliest  deployment  opportunity 
and  other  metrics  such  as  cost  may  be  the 
overriding  factor.  Either  way  the  final  result  can 
be  used  to  construct  a  preliminary  block  upgrade 
plan  corresponding  to  the  time  epochs  where 


SPI  Normalized 


in 


Figure  4.  Normalized  SPI  Plot  in  Cost-Time  Space 


incur.  Technologies  that  meet  or  exceed  the 
threshold  level  of  performance  are  selected  over 
those  that  do  not,  provided  they  fall  within  the 
maximum  cost  constraint.  Technologies  that 
satisfy  both  of  these  constraints  are  reordered  in 
descending  order  according  to  their  ROI  value  - 
highest  ROI  to  lowest. 

The  next  step  is  to  look  at  the  future  time  epochs 
where  the  various  technologies  are  projected  to 
be  ready  for  insertion.  Those  technologies  that 
counter  projected  threats  with  acceptable 
performance,  are  within  the  maximum  cost 
constraint,  and  fall  within  or  precede  time 
epochs  in  which  the  projected  threats  are  viable 
represent  the  best  choices.  Therefore,  we  rank 
the  technologies  in  descending  order  (highest  to 
lowest)  according  to  ROI  value  (that  exceeds  the 
minimum  performance  threshold  and  within  the 
maximum  cost  constraint)  that  have  TOIs,  which 


capabilities  against  specific  threats  are  projected 
to  be  available. 

Implementing  the  Methodology 

The  SPI  methodology  can  be  implemented  with 
tools  that  are  available  today.  An  overview  of 
the  tools  and  process  for  calculating  SPI  and  R 
is  shown  in  Figure  5.  One  system  model  that 
can  be  used  to  support  the  implementation  is  the 
Quick  Defended  Area  (QDA)  model,  which 
models  the  system  architecture  and  elements, 
their  performance  parameters,  and  the  system 
response  to  the  threat.  QDA  has  been  used 
extensively  in  the  Ballistic  Missile  Defense 
System  Architecture  Studies  (BMDSAS)  and  to 
support  the  assessment  of  technologies  for  the 
Joint  Center  for  Technology  Integration  (JCTI). 
As  such,  it  is  well  proven  and  has  a  large 
repertoire  of  models  for  architectural  elements 
and  threats  suites. 
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Calculating  Normalized  SPI  using  QDA  &  PZL  Calculator 
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Figure  5.  Implementation  of  Process  for  Calculating  SPI  and  R 


QDA  divides  the  defended  area  into  a  grid 
specified  by  the  user,  flies  the  threats,  and 
computes  the  engagement  opportunities 
associated  with  each  cell  within  the  grid,  based 
upon  the  deployment  physical  constraints  and 
the  CPPs  for  the  elements  of  the  system 
architecture.  The  outputs  are  engagement 
opportunities  and  resources  utilized.  CPPs  may 
be  calculated  using  the  analytical  models  from 
the  Analysis  Tools  developed  by  Haight10  or 
other  available  models  using  characteristics  of 
the  technologies.  Engagement  opportunities 
are  input  to  the  PZL  Calculator,  which  computes 
the  PZL  associated  with  each  cell  within  the 
defended  area.  Cells  can  take  on  various 
values,  Vi,  corresponding  to  the  system 
objective.  They  might  take  on  a  uniform  value  of 
1  for  all  cells  if  we  are  only  interested  in  zero 
leakage;  the  population  within  the  cell  might  be 
used  in  the  case  of  population  defense;  or  the 
value  of  critical  military  targets  within  the  cells 
might  be  used  for  defense  of  military  assets. 
The  SPI  Calculator  computes  the  normalized 
SPI  from  the  Pzu  and  V,  values. 


Several  cost  models  are  available  that  could 
support  the  requirements  of  this  methodology. 
The  primary  requirement  is  that  the  model  be 
able  to  compute  the  RDT&E,  acquisition,  and 
O&S  costs  using  resource  requirements  and 
characterizations  of  the  technologies.  For 
mature  system  concepts  the  system  LCC 
models  could  be  used.  For  advanced  concepts, 
other  models  may  offer  greater  flexibility.  In 
some  cases  it  may  be  necessary  to  use  several 
element  cost  models,  such  as  the  GBI  Cost 
Model2,  to  obtain  the  desired  results.  To  ensure 
consistency  across  the  full  range  of  concepts 
and  technologies,  a  standard  cost  model  should 
be  adopted  to  support  this  requirement.  A 
simple  ROI  Calculator  can  be  used  to  support 
the  ranking  methodology. 

The  TRL  methodology  is  well  defined  and  can 
be  directly  applied  to  characterize  the  maturity  of 
the  technology.  Today  it  is  a  manual  process  to 
estimate  the  time  epoch  when  the  technology  is 
ready  for  insertion;  therefore,  the  TOI  must  be 
estimated  based  on  inputs  from  the  technologist. 
A  model  to  estimate  TOI  based  on  the  current 
status  of  the  technology,  the  technology  type 
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Table  1.  Assessment  and  Ranking  of  Technology  Options 


Measure 

Value 

Baseline 

A 

B 

C 

D 

E 

SPI 

Population 

490.5 

466.4 

585.0 

617.8 

680.9 

689.1 

SPI 

Uniform 

4649 

4492 

5511 

6926 

7678 

7786 

SPI  Normalized 

Population 

0.65 

0.61 

0.77 

0.81 

0.90 

0.91 

SPI  Normalized 

Uniform 

0.48 

0.47 

0.57 

0.72 

0.80 

0.81 

^Normalized 

$Bk/$BBL 

1.00 

0.76 

0.56 

0.91 

1.06 

1.10 

ROI 

SPI/C 

0.65 

0.80 

1.37 

0.90 

0.84 

0.83 

TRL 

Number 

6 

5 

4 

4 

4 

3 

TOI 

Time 

Epoch 

T 1 

T 1 

T 1 

T2 

T2 

T2 

Ranking 

3 

2 

1 

1 

2 

3 

and  other  technology  unique  factors  is  needed 
to  support  the  methodology  in  the  future. 
Techniques  using  empirical  data  such  as  those 
employed  in  software  estimating  tools  (e.g., 
COCOMO  or  REVIC)  may  offer  fruitful  avenues 
for  successful  implementation  of  such  a  model  in 
the  future. 

Application  of  the  Methodology 

The  methodology  has  been  applied  to  assess 
and  rank  five  technologies  considered  for 
defense  of  Europe11.  A  summary  of  the 
measures  for  the  baseline  and  each  technology 
concept  is  provided  in  Table  1.  The  SPI  is 
calculated  for  two  Value  metrics  —  one  based 
on  population12  within  the  cell  and  a  second  that 
assumes  all  cells  are  of  uniform  value.  Note, 
different  defended  area  values  yield  different 
values  for  SPI  metrics.  Also  we  have  not 
included  R  because  it  is  beyond  the  scope  of 
this  paper. 

Ranking  by  ROI  yields  B,  C,  D,  E,  and  A,  all  with 
better  ROI  than  the  Baseline.  Assuming  a 
minimum  acceptable  SPI  of  0.65,  only  B,  C,  D, 
and  E  are  solutions.  Adding  a  maximum  cost 
constrain  of  1.0  reduces  the  solution  set  to  B 
and  C.  Under  these  constraints  B  is  the  best 
choice.  In  this  example  the  technologies  tend  to 
fall  into  two  time  epochs,  T1  and  T2.  If  a 
minimum  acceptable  SPI  of  0.9  is  desired  and  a 
capability  is  needed  before  T2,  we  might 
consider  implementing  B  in  time  epoch  T1  and  D 
in  time  epoch  T2,  assuming  cost  constraints  can 
be  relaxed.  Otherwise,  none  of  the  technologies 
offer  a  viable  solution  within  the  constraints. 

Conclusions  and  Recommendations 

A  methodology  using  a  System  Performance 
Index  for  assessing  and  ranking  BMD 
technologies  has  been  outlined.  This 


methodology  addresses  the  questions  of  system 
performance,  cost,  and  availability  for  insertion. 
It  is  objective,  based  on  the  physics  of  the 
problem,  and  avoids  the  problems  encountered 
with  subjective  weighting  schemes.  The 
methodology  completely  characterizes  the 
technology  with  respect  to  a  baseline  using  the 
SPI,  R,  C,  and  ROI  measures  and  utilizes  the 
TRL  and  TOI  measures  to  quantify  readiness.  It 
can  quantify  the  effects  of  multiple  technologies, 
as  well  as  small  effects  by  a  technology  in 
multiple  areas.  Further,  it  provides  a  framework 
for  systematic  tradeoffs  between  performance, 
cost,  and  time  of  insertion. 

The  SPI  and  R  can  be  calculated  using  proven 
and  accepted  tools,  which  are  available  today, 
with  only  minor  additions.  It  is  recommended 
that  a  standard  set  of  tools  be  adapted  for  all 
BMD  technology  assessment  and  prioritization 
work. 

The  methodology  requires  a  cost  model  to 
compute  total  ownership  cost.  The  system  LCC 
models,  element  cost  models,  or  other  accepted 
cost  models  could  be  used  to  support  this 
methodology.  It  is  recommended  that  a 
standard  cost  model  be  adopted  to  support  this 
methodology  to  ensure  consistency  of  results. 

The  NASA  methodology  for  determining  the 
technology  readiness  level,  which  has  been 
adopted  by  the  Army,  provides  a  useful  measure 
of  where  the  technology  is  in  its  developmental 
cycle,  but  is  not  sufficient.  The  TOI,  which 
answers  the  important  question  of  when  the 
technology  will  be  ready  for  insertion,  is  today  a 
manual  process  that  relies  entirely  on  the 
technologist’s  input.  It  is  recommended  that  a 
model  be  developed  and  validated  to  predict 
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when  the  technology  will  be  available  for 
insertion. 
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